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Executive Summary 


Motivation is a central topic in educational psychology. A multitude of studies have shown that 
students’ motivation affects their learning behaviors and ultimately their learning outcomes— 
i.e., academic achievement—even after taking into account students’ cognitive ability, previous 
academic achievement, and other demographic and social characteristics (Schunk, Pintrich, and 
Meece 2008; Wigfield et al. 2006; Zhang et al. 2017; Bohrnstedt et al. 2020). Student learning 
largely relies on processing and ultimately comprehending written materials which highlights the 
importance of reading for students’ academic success. Hulme and Snowling (2013) suggest that 
“learning to read is a key objective of early education and difficulties in learning to read have 
serious adverse consequences.” Proficient reading skills play a significant role in students’ 
success in every academic domain (Guthrie et al. 2000; Schiefele et al. 2012). Therefore, reading 
is considered a precursor to students’ academic achievement, and fostering students’ motivation 
to read is a key issue for educators. 


National assessment data on reading performance are concerning in that they indicate that many 
of the nation’s students are not doing well in reading. For example, in 2017, the National 
Assessment of Educational Progress (NAEP) ! found that roughly two thirds of 4 and 8" grade 
students performed below the NAEP Proficient achievement level in reading (63 percent and 64 
percent, respectively). Furthermore, 32 percent of 4" grade students and 24 percent of 8" grade 
students performed below the NAEP Basic achievement level, indicating that many students are 
struggling to master even fundamental reading skills. 


Differences in reading performance by student subgroups are concerning as well. The 
achievement gaps between White and Black students and between White and Hispanic students 
are substantial. For example, the percentage of White 8" graders who scored at or above the 
NAEP Proficient achievement level in reading in 2017 (45 percent) was double the percentages 
of Black (18 percent) and Hispanic (23 percent) students performing at or above NAEP 
Proficient. In addition, 16 percent of White 8" graders performed below the NAEP Basic 
achievement level in reading in 2017, while the percentages of Black and Hispanic 8" graders 
scoring below NAEP Basic were 40 and 33 percent, respectively. 


Understanding the relationship between reading motivation and reading achievement helps shed 
light on the underlying psychological processes affecting students’ reading behavior and 
achievement which in turn can inform classroom instructional practices to promote students’ 
reading motivation. Although many studies demonstrate the importance of reading motivation on 
reading achievement, several gaps in the research literature remain. For instance, most of the 
literature on reading motivation focuses on students in elementary school and less frequently on 
students in middle school (Gottfried et al., 2001; Kelley and Decker 2009). Additionally, many 
reading motivation studies are based on relatively small sample sizes decreasing the 
generalizability of results. Also, the effects of reading motivation in many studies are tarnished 
by the influences of confounding variables (e.g., student socioeconomic status (SES), student 


1 NAEP is the largest nationally representative assessment of U.S. students’ performance in a variety of academic subjects, 
including reading, mathematics, science, and social studies. 


cognitive abilities) (Schiefele et al. 2012) rendering it difficult to identify the independent effect 
of reading motivation on reading achievement. 


This study aims to understand the role that reading motivation plays in middle school reading 
achievement (including achievement gaps) by analyzing the 2015 grade 8 NAEP reading data. 
The current study focuses on identifying the unique effects of student-level reading motivation 
and aggregated school-level mean reading motivation on reading achievement. Hierarchical 
linear modeling is used to partition variability in student reading achievement into within- and 
between-school components after student- and school-level demographic variables are taken into 
account. In addition, this study investigates whether the identified unique effects of reading 
motivation on reading achievement vary by gender, race/ethnicity, English language learner 
status (ELL), and other demographic variables. Finally, the study examines the roles that 
students’ demographic characteristics and reading motivation play in understanding reading 
achievement gaps based on students’ race/ethnicity and ELL status, 


A summary of the results and discussions of the findings are presented below: 


¢ The analyses indicated that student reading motivation is substantially associated with 
student reading achievement even after taking into account student gender, race/ethnicity, 
and SES, ELL, Individualized Education Program (IEP) statuses, and interaction terms. 
Each unit difference in student reading motivation (reading motivation scale ranging 
from 0 to 3) was predicted to result in a 10.67-point difference in NAEP reading scores, 
which is approximately one third of a standard deviation. 


¢ Another important finding of the current study is the unique and substantial effect of 
school-level mean motivation on school mean reading achievement even in the presence 
of other school demographic variables including mean school-level SES, proportion of 
racial/ethnic minority students, and proportion of IEP students. The estimated coefficient 
for school mean motivation suggests that for each unit difference in school mean 
motivation, there is a predicted 17-point difference in school mean reading achievement 
which is about half of one standard deviation. This result highlights a school level 
contextual effect as well. That is, the reading motivation of one’s peers in school matters 
substantially as well. Schools might be able to promote students’ academic development 
not only directly through effective teaching but also indirectly through a school climate 
that fosters reading motivation. 


¢ Variables identified in the HLM model were able to explain a substantial amount of the 
reading achievement gap between White and Black students, but not all of it. The original 
achievement gap between White and Black students was 25 points (0.74 SD units) for 
reading achievement. After taking into account student reading motivation, student 
demographic variables, and the interaction terms, the remaining unexplained achievement 
gap was 11 points (0.32 SD units), which is still statistically significant but substantially 
smaller than the observed reading gap. 
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¢ The achievement gap between White and Hispanic students was 20 points, which is 0.64 
standard deviation (SD) units, before taking into account other explanatory variables. 
After accounting for student reading motivation, student demographic variables, and the 
interaction terms, the remaining unexplained achievement gap was less than 1 point and 
no longer statistically significant. 


¢ The observed achievement gap between ELL and non-ELL students was 43 points (1.26 
SD units). After taking into account student motivation and social demographic 
background, ELL students scored 20 NAEP points (0.60 SD units) lower compared to 
non-ELL students. Also, this study investigated interactions among motivation, student 
ELL status, and being Hispanic students. The NAEP reading scores for white and 
Hispanic students after taking into account their ELL status and their socio-demographic 
backgrounds were not statistically different. This suggests that the reading achievement 
gap between white and Hispanic students is largely due to differences between ELL and 
non- ELL students rather than being Hispanic. 


¢ The interaction between ELL status and reading motivation was statistically significant: 
specifically, the effect of motivation on reading achievement was substantially weaker for 
ELL students than for non-ELL students. Overall, students having a higher level of 
reading motivation on average were more likely to do well on a reading achievement 
assessment than those with lower reading motivation. However, for ELL students, the 
effects of motivation seem muted apparently due to the absence of a strong grasp of the 
English language. 


The current study results found that student reading motivation is substantially associated with 
student reading achievement, even after taking into account student gender, race/ethnicity, and 
SES, ELL, IEP statuses, and interaction terms, which comports with the literature on the effects 
of reading motivation on reading achievement. In addition, the current study identifies the unique 
and substantial effect of school-level mean motivation on school mean reading achievement even 
in the presence of other school demographic variables, which highlights the significant role 
schools might have on promoting students’ academic development. 
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Introduction 


Motivation is a central topic in educational psychology. A multitude of studies have shown that 
students’ motivation affects their learning behaviors and ultimately their learning outcomes—.e., 
academic achievement—even after taking into account students’ cognitive ability, previous 
academic achievement, and other demographic and social characteristics (Schunk, Pintrich, and 
Meece 2008; Wigfield et al. 2006; Zhang et al. 2017; Bohrnstedt et al. 2020). Student learning 
largely relies on processing and ultimately comprehending written materials which highlights the 
importance of reading for students’ academic success. Hulme and Snowling (2013) suggested that 
“learning to read is a key objective of early education and difficulties in learning to read have 
serious adverse consequences.” Proficient reading skills play a significant role in students’ success 
in every academic domain (Guthrie et al. 2000; Schiefele et al. 2012). Therefore, reading is 
considered a precursor to students’ academic achievement, and fostering students’ motivation to 
read is a key issue for educators. 


Unfortunately, national assessment data on reading performance are concerning. The National 
Assessment of Educational Progress (NAEP) assesses student performance in reading at grades 4, 
8, and 12 in both public and private schools every two years across the nation.! NAEP 
achievement levels define what students should know and be able to do: NAEP Basic indicates 
partial mastery of fundamental skills, and NAEP Proficient indicates demonstrated competency 
over challenging subject matter. In 2017, roughly two thirds of 4" and 8" grade students 
performed below the NAEP Proficient achievement level in reading (63 percent and 64 percent, 
respectively). Furthermore, 32 percent of 4" grade students and 24 percent of 8" grade students 
performed below the NAEP Basic achievement level, indicating that many students are struggling 
to master even fundamental reading skills. 


Differences in reading performance by student subgroups are concerning as well. The achievement 
gaps between White and Black students and between White and Hispanic students are substantial. 
For example, the percentage of White 8" graders who scored at or above the NAEP Proficient 
achievement level in reading in 2017 (45 percent) was double the percentages of Black (18 
percent) and Hispanic (23 percent) students performing at or above NAEP Proficient. In addition, 
16 percent of White 8" graders performed below the NAEP Basic achievement level in reading in 
2017, while the percentages of Black and Hispanic 8" graders scoring below NAEP Basic were 40 
and 33 percent, respectively. 


Understanding the relationship between reading motivation and reading achievement helps shed 
light on the underlying psychological processes affecting students’ reading behavior and 
achievement and may inform classroom instructional practices to promote students’ reading 
motivation. Although many studies demonstrate the importance of reading motivation on reading 
achievement, several gaps in the research literature remain. For instance, most of the literature on 
reading motivation is about students in elementary school, and less frequently focuses on students 
in middle school (Gottfried et al. 2001; Kelley and Decker 2009). Additionally, many reading 
motivation studies are based on relatively small sample sizes, decreasing the generalizability of 


‘NAEP is the largest nationally representative assessment of U.S. students’ performance in a variety of academic subjects, 
including reading, mathematics, science, and social studies. 


results. Also, the effects of reading motivation in many studies are tarnished by the influences of 
confounding variables (e.g., student SES, student cognitive abilities) Schiefele et al. 2012), 
rendering it difficult to identify the independent effect of reading motivation on reading 
achievement. 


This study aims to understand the role that reading motivation plays in middle school reading 
achievement (including achievement gaps) by analyzing the 2015 grade 8 NAEP reading data. 
The current study focuses on identifying the unique effects of student-level reading motivation 
and aggregated school-level mean reading motivation on reading achievement. Hierarchical 
linear modeling is used to partition variability in student reading achievement into within- and 
between-school components after student- and school-level demographic variables are taken into 
account. In addition, this study investigates whether the identified unique effects of reading 
motivation on reading achievement vary by gender, race/ethnicity, English language learner 
status (ELL), and other demographic variables. Meanwhile, the current study strives to 
understand the reading achievement gaps based on students’ race/ethnicity and ELL status. The 
study examines how students’ demographic characteristics and reading motivation explain 
White-Black, White-Hispanic, and ELL and non-ELL students’ achievement gaps. To achieve 
these aims this study addresses the following six research questions: 


1. Is reading motivation significantly associated with NAEP reading achievement? 


2. Does the association between reading motivation and reading achievement persist even 
after student social demographic characteristics are taken into account? 


3. Does the association investigated in research question 2 between motivation and 
achievement vary by gender, race/ethnicity, socioeconomic status, English language 
learner status, and individualized education program status? In addition, does the 
association vary across schools? How are student-level characteristics including 
motivation and demographics helping to explain White-Black, White-Hispanic and ELL- 
non-ELL students’ achievement gaps? 


4. Does school-level mean reading motivation explain school-level variation in student 
reading achievement? 


5. Do school-level characteristics (social demographics) explain school-level variation in 
student reading achievement? 


6. Do school-level characteristics help explain the association between student reading 
motivation and NAEFP reading scores? 


Theoretical Background 


Motivation can be thought of as the driver or energy expended in pursuit of a goal (reading 
achievement in this study). It can either be extrinsic (based on actual or perceived rewards or 
punishments) or intrinsic (based on internalized beliefs and values about the importance of 
achieving the goal) (Deci 1975). More generally, motivation has been defined as “the process 
whereby goal-directed activity is instigated and sustained” (Pintrich and Schunk 2002, p. 5). 


Because one cannot directly observe this internal, latent process, a theoretical framework is 
necessary for measuring motivational beliefs. And as noted by Lameva and Choneteva (2013): 
“Due to the latency of psychological constructs, the construct motivation can be conceptualized 
in different ways, with different theories focusing on different psychological processes.” (p. 4) 


Contemporary literature on achievement motivation is grounded in a social cognitive 
perspective, focusing on understanding students’ cognitive process of whether they choose to 
engage in learning tasks and how persistent they are while working on those tasks. Central 
motivational beliefs include individuals’ competence-related beliefs such as self-efficacy and 
subjective values (Eccles and Wigfield 2002). Competence-related beliefs refer to beliefs about 
individuals’ current and future perceptions of the confidence they have of how well they can or 
will perform different tasks and activities currently or in the future; values refer to incentives or 
reasons for undertaking the task or activity. More specifically, motivational theories address two 
questions: (1) Can I do the task? and (2) Why am I doing the task? For example, in the domain of 
reading, competence-related beliefs address individuals’ beliefs on whether they can do well in 
reading. Values explain why individuals want to read or take on more challenging reading topics, 
or participate in other reading activities like book clubs. 


Also, academic motivation is subject specific. In the domain of reading motivation, defined “as 
the individual's goals and beliefs with regard to reading” (Guthrie and Wigfield 1999), the focal 
point is the second question—why students engage in reading and related activities. More 
specifically, reading motivational theorists distinguish intrinsic and extrinsic motivation to 
explain students’ reading behaviors and related reading achievement outcomes (e.g., reading test 
scores and reading skills) (Schiefel et al. 2012). Intrinsic motivation arises from students’ own 
self-expressed interests, and extrinsic motivation is from outside influences such as teacher and 
parent expectations, perceived rewards, and grades. 


Modern motivation theories not only consider motivation as an individual characteristic, but also 
acknowledge the important impacts of social context and interpersonal relations on motivation. 
The current study uses a modern motivational framework to conceptualize reading motivation 
and its impacts on reading achievement. Intrinsic and extrinsic reading motivation beliefs are the 
major components investigated in this study. Student family and school background variables are 
included to capture the social context so as to better understand the relationships between reading 
motivation and achievement. 


Intrinsic and Extrinsic Reading Motivation 


The most widely known line of research in reading motivation pertains to the distinction between 
intrinsic and extrinsic motivation. Intrinsic motivation is defined as “the willingness to read 
because that activity is satisfying or rewarding in its own right...Extrinsic motivation refers to 
reasons for reading that are external to both the activity of reading and the topic of the text” 
(Schiefel et al. 2012, p. 429). In other words, students with high intrinsic reading motivation are 
actively engaged in reading or reading-related activities due to the pure pleasure and enjoyment 
that arise from reading. However, students with high extrinsic reading motivation participate in 
reading or reading-related activities for reasons not inherent in the reading itself—instead they 
do it for reasons such as earning a reward. For example, a middle school student may be 
motivated to read because of his/her interest or curiosity in a particular topic, making their 


motivation intrinsic. While a student who reads to complete a school assignment or to get high 
scores in a particular subject is extrinsically motivated. 


Researchers have conducted both qualitative and quantitative studies to investigate the 
dimensions of intrinsic and extrinsic reading motivation. Although there is not unanimous 
agreement on the specific dimensions of intrinsic and extrinsic reading motivation, many reading 
motivation theorists agree that intrinsic reading motivation includes the dimensions of curiosity, 
enjoyment/interest, and involvement; that is, they focus on incentives within the reading process 
itself (Wigfield and Guthrie 1997; Guthrie and Wigfield, 1999; Wang and Guthrie 2004). 
Curiosity is the desire to learn more about personally interesting topics. Enjoyment/interest 
emphasizes the pure fun and excitement of reading. Involvement refers to getting lost in a story, 
and experiencing imaginative actions. Meanwhile, extrinsic motivations include competition, 
utility, recognition, and social purpose, emphasizing the incentives that represent consequences 
of reading. Competition refers to a desire to reach higher levels of reading achievement than that 
reached by other students. Utility is about why students value reading, for instance that reading is 
important because it helps them learn. Social purpose refers to students’ goal of reading to 
maintain their social connections. 


Although researchers distinguish reading motivation as intrinsic and extrinsic, “the intrinsic and 
extrinsic factors of reading motivation tend to be positivity correlated, thus indicating that 
reading is usually simultaneously—but to varying degrees—motivated by intrinsic and extrinsic 
incentives” (Schiefel et al. 2012, p. 437). Therefore, intrinsic and extrinsic motivation should not 
be studied apart from one another. As a result, the current study investigates both intrinsic and 
extrinsic reading motivation, including the dimensions of curiosity, enjoyment/interest, social 
purpose, and utility. 


The Effect of Reading Motivation on Reading Achievement 


Not surprisingly, research shows that students’ intrinsic motivation plays a significant role in 
their academic achievement and learning behavior. Gottfried (1985) investigated how student 
academic intrinsic motivation affects students’ academic achievement in reading, mathematics, 
and social studies for students from grade 4 to grade 8. Intrinsic motivation was measured as 
students’ enjoyment of learning, master-orientation, curiosity, and persistence, using items such 
as “I enjoy learning new things,” “I like to do as much work as I can,” “I enjoy doing hard 
assignments,” and “I like to find answers to questions.” The study found that students’ academic 
intrinsic motivation was significantly and positively correlated with students’ school academic 
achievement across all examined subject areas. Wang and Guthrie (2004) studied the relationship 
between reading comprehension and intrinsic and extrinsic motivation among 4" grade students 
in the U.S. and China. Intrinsic motivation was measured as student interest and enjoyment in 
reading as well as their preference of challenge. The structural equation modeling path analyses 
found that intrinsic motivation had a larger relationship with reading comprehension than did all 
other motivation constructs measured and cognitive factors in the model, including extrinsic 
motivation and prior reading achievement. Retelsdorf, Koller, and Moller (2011) drew upon a 
longitudinal sample of grade 5 to grade 8 students to investigate how students’ intrinsic and 
extrinsic reading motivation related to their reading performance and growth. Reading intrinsic 
motivation was defined as reading for enjoyment (e.g., “I enjoy reading books”’) and reading for 
personal interest (e.g., “I read to learn about my topics of interest”), while reading extrinsic 


motivation was defined as reading for competition. Student cognitive ability was also included in 
the study model. The latent growth curve modeling results showed that intrinsic reading 
motivation was significantly positively related to student reading performance. More 
importantly, intrinsic reading motivation strongly predicted academic growth in reading, even 
after taking into account student socio-demographic background and previous academic 
achievement. 


Overall, the literature consistently demonstrates that intrinsic motivation is positively associated 
with reading achievement, particularly the dimensions of enjoyment/interest and curiosity. In 
contrast, extrinsic reading motivation is less studied and the results for the relationship between 
extrinsic reading motivation and reading achievement are mixed. Baker and Wigfield (1999) 
found that students’ extrinsic motivation—measured using recognition, competition, and 
grades—was positively associated with reading performance. Lau and Chan (2003) reported 
similar findings by investigating 159 Chinese students in Hong Kong. Both intrinsic and 
extrinsic motivation had significantly positive correlations with reading scores. Andreassen and 
Braten (2010) assessed the relationship between intrinsic and extrinsic reading motivation and 
reading strategies and reading performance. They found positive and significant associations 
between intrinsic motivation and reading performance. However, the associations between 
extrinsic motivation and reading performance were not significant. They proposed that the non- 
significant effect of extrinsic motivation might be due to suppression effects since intrinsic 
motivation and extrinsic motivation had a positive correlation with each other. Therefore, 
creating a reading motivation composite containing both intrinsic and extrinsic motivation might 
be a good alternative. 


Effects of Reading Motivation on Reading Achievement across Student Subgroups 
Gender, reading motivation, and reading achievement 


Gender plays a significant role in shaping students’ types of achievement motivation and thus 
influence achievement behaviors (e.g., participating in a book club; taking an advanced 
mathematics course) and related outcomes. 


In contrast to mathematics, where male students have been found perform modestly better than 
females and have more positive competence, identity beliefs, and subjective values toward 
mathematics than female students (Eccles et al. 1993; Zhang et al. 2017; Bohrnstedt et al. 2020), 
various national and international assessments have found that female students outperform their 
male peers in reading. For example, NAEP data over the past decade have consistently shown 
sizable reading gender disparities favoring female students at 4", 8", and 12™ grades, with 
roughly a 10-point difference on a 500-point scale with standard deviations of around 30 points. 
Gender gaps in reading achievement have formed the basis for arguments that schooling unfairly 
favors female students academically (Robinson and Lubienski 2011). Also, female students 
typically report higher reading motivation than male students (Baker and Wigfield 1999; 
Mucherah and Yoder 2008; Wigfield and Guthrie 1997). McGeown (2015) investigated gender 
differences in reading choices, reading motivation, and reading skills. There were no significant 
gender differences in reading skills. However, girls reported significantly higher levels of 
curiosity and involvement (intrinsic motivation), but not in challenge (extrinsic motivation). 
Durik, Vida, and Eccles (2006) found that girls valued reading and English more than did boys 
in terms of intrinsic value and importance at both 4th and 10th grade. 


In addition, research has indicated that motivational beliefs decline over the course of schooling, 
but the rate of decline varies by gender and subject (Wigfield and Eccles 2000; Wigfield et al. 
1997). Jacobs et al. (2002) found that male students’ competence beliefs and value of reading 
declined more rapidly than did female students’; this might be one of several factors explaining 
the increasing gender gap in reading achievement across grade levels. Wigfield, Gladstone, and 
Turci (2016) suggests that these gender difference findings for reading motivation and reading 
achievement are reflections of stereotype expectations—female students are more interested in 
and positive about reading compared to male students—and called for more research focusing 
on improving male students’ reading motivation alongside current efforts to promote female 
students’ engagement and involvement in mathematics, science, and engineering. 


Race/ethnicity, reading motivation, and reading achievement 


Educational achievement gaps between White students and their Black and Hispanic peers 
contribute to racial/ethnic economic inequality (Jencks and Phillips 1998). The recent data from 
NAEP show significant achievement gaps in reading between White students and Black and 
Hispanic students. In 2015, White students performed about 0.76 and 0.71 standard deviations 
higher than Black and Hispanic students, respectively, in the grade 8 reading assessment. Similar 
gaps were observed for grade 4 and grade 12. 


Students’ racial/ethnic background also plays a role in shaping their reading motivation. Most 
reading motivation studies have found that Black students have higher positive academic self- 
concepts, a more positive academic self-efficacy, and stronger academic beliefs than White 
students even when they were doing less well in school (Baker and Wigfield 1999; Graham 
1994; Graham and Taylor 2002; Zhang et al. 2018). For example, Baker and Wigfield (1999) 
found that African American students in 5" and 6" grade report higher reading motivational 
beliefs than White students on 9 out of the 11 constructs measured in the study. African 
American students had higher self-efficacy, intrinsic motivation, and valuing of reading scores 
than White students. In addition, Jerrim and Schoon (2014) also found that Asian students were 
often modest in their competence beliefs despite their generally high achievement. 


However, the correlation between reading competency beliefs and reading achievement was 
lower among African American and Hispanic students than among White and Asian students 
(Graham 1994; Graham and Taylor 2002; Unrau and Schlackman 2006). Unrau and Schlackman 
(2006) in a study of Asian and Hispanic middle school students living in an urban, 
socioeconomically disadvantaged area, found that intrinsic reading motivation was positively 
associated with reading achievement for Asian students, but was not significant for Hispanic 
students. Guthrie, Coddington, and Wigfield (2009) also investigated ethnic differences in the 
effects of reading motivation on reading achievement for African American and White students. 
They found that intrinsic motivation was not significantly related to reading comprehension for 
African American students, but was significant for White students. 


All in all, the relationship between motivation and academic performance varies across different 
racial/ethnic groups. Wigfield, Gladstone, and Turci (2016) suggest that the extent to which 
race/ethnicity affects the relationship between reading motivation and reading should be 
explored further. 


The current study not only investigates the relationship between reading motivation and reading 
achievement, but also aims to test whether the relationship varies by student gender, 
race/ethnicity, and other social demographics. Such analyses will help policymakers and 
educators develop effective educational practices based on students’ demographic characteristics. 


Social Context, Motivation, and Achievement 


Students’ motivational beliefs develop under the influence of various social contexts, including 
family and school. Therefore, a better understanding of students’ motivational beliefs as they 
relate to reading performance requires having knowledge of family and school contexts. 


> 


Data from the Wisconsin Longitudinal Study showed a significant relationship between students 
educational aspirations and their parents’ educational attainment (Sewell and Shah 1968). 
Previous research has also shown that family background affects students’ reading achievement. 
Parents’ educational level and their occupational status were related to their children’s reading 
and other educational outcomes (Yeung, Linver, and Brooks-Gunn 2002). Retelsdorf, Koller, 
and Moller (2011) suggest that “the mode of action in the association between socioeconomic 
status (SES) and students’ outcomes deals with the family’s ability and willingness to invest in 
providing a stimulating learning environment. One particular aspect of learning environments 
deals with the variability of literary resources on hand to a child. Thereby, the number of books a 
family possesses has been proven as a valid indicator of such a home literacy environment” (Elly 
1992, p. 551). Overall, family income and parents’ education relate to students’ reading 
motivation and achievement as well as to the reading achievement gaps between racial/ethnic 
minority students and White students. 


School context is also an important factor in understanding student achievement. Freiberg (1999) 
has argued that school environment influences student achievement through student attachment, 
commitment, involvement, and, most importantly, through schools’ resources and academic 
climate. Perry and McConney (2010) found that school socioeconomic status (SES) was 
significantly associated with students’ academic achievement. Rumberger and Parlardy (2005) 
used the National Education Longitudinal Survey of 1988 to examine individual and school 
effects on achievement growth between grade 8 and grade 12 in mathematics, science, reading, 
and history. They found that school-level SES had as much impact on students’ achievement as 
the students’ individual-level SES did, after taking into account other background factors. 


The current study employs a social cognitive approach to conceptualize reading motivation and 
its relationship with reading achievement. The development of sequential hierarchical linear 
models focuses on identifying unique reading motivation effects by taking into account student- 
and school-level demographic variables. 


Methods 


Data 


This study uses grade 8 2015 National Assessment of Educational Progress (NAEP) reading 
assessment data, based on a nationally representative sample of 139,000 students from 7,500 
schools. Because the current study’s focus is on public school students, students from private 
schools, Bureau of Indian Education schools, and Department of Defense schools were excluded 
from the sample, leaving a sample of 135,000 students from 5,670 public schools. In addition, 
around 20 percent of students had missing demographic or reading motivation information 
needed for the hierarchical linear model (HLM) analysis, so were also excluded from the final 
analytic sample. As a result, the final analytic sample contained roughly 105,000 students from 
4,530 schools. Table 1 shows the distribution of the final analytic sample by sex, race/ethnicity, 
and other social demographic variables. 


Table 1. Basic demographic characteristics for the final analytic sample 


Final analytic sample 
Unweighted number of Unweighted Weighted percentage! 
Student demographic characteristics students percentage Percentage | Standard error 
Total 105,000 100.00 100.00 t 
Gender 
Male 53,400 50.87 50.66 0.15 
Female 51,600 49.13 49.34 0.15 
Race/ethnicity 
White 57,900 55.14 52.30 0.48 
Black 17,300 16.47 14.50 0.35 
Hispanic 20,200 19.19 24.57 0.47 
Asian/Pacific Islander 5,200 4.92 5.24 0.22 
American Indian/Alaska Native 1,900 1.78 0.94 0.06 
Other 2,600 2.50 2.44 0.06 
NSLP eligibility 
Eligible 54,300 51.71 51.43 0.49 
Not eligible 50,000 47.57 47.56 0.50 
Information not shown 770 0.72 1.02 0.21 
English language learner 
Yes 5,400 5.19 6.03 0.19 
No 99,500 94.81 93.97 0.19 
Missing 0 0.00 0.00 0.00 
Individualized Education Program 
Yes 12,400 11.80 11.88 0.12 
No 92,600 88.20 88.12 0.12 
Missing 0 0.00 0.00 0.00 
+ Not applicable. 


' Weighted using grade 8 NAEP survey weight. 


NOTE: NSLP=National School Lunch Program. Detail may not sum to totals because of rounding. Numbers are rounded to the 
nearest 100 or 10 based on the NCES technical standards. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational 


Progress. 


Tables 2 and 3 display the comparison of the final analytic sample to the original NAEP public 
school sample. The results show that these two samples are very similar in terms of basic 
demographic characteristics. Compared to the NAEP public school sample, the final analytic 


sample has a slightly higher percentage of White students. The percentage of students who were 
eligible for the National School Lunch Program (NSLP), were English language learners (ELLs), 
and had an Individualized Education Program (IEP) were very similar between the final analytic 
sample and the NAEP public school sample, with a less than | percentage point difference in 
each variable (except the IEP with around 1.5 percentage point difference). In addition, the 
NAEP reading performance of the final analytic sample was very similar to that of the original 
NAEP public school sample. As expected, the standard errors of the final analytic sample were 
slightly larger in all cases than those in the NAEP public school sample since the analytic sample 


is smaller. Overall, no indication that missing data biased the study’s results was detected from 


the various analyses conducted to examine this issue’. 


Table 2. Comparison of the final analytic sample with the 2015 grade 8 public school 


sample, by student subgroups 


NAEP public school 
Final analytic sample sample 
Weighted | Standard Weighted | Standard 

Student demographic characteristics Percentage error | Percentage error 
Total 100.0 t 100.0 t 
Gender 

Male 50.66 0.15 51.04 0.10 

Female 49.34 0.15 48.96 0.10 
Race/ethnicity 
White 52.30 0.48 50.53 0.40 
Black 14.50 0.35 15.30 0.28 
Hispanic 24.57 0.47 25.13 0.38 
Asian/Pacific Islander 5.24 0.22 5.62 0.22 
American Indian/Alaska Native 0.94 0.06 1.04 0.05 
Other 2.44 0.06 2.39 0.05 
NSLP eligibility 

Eligible 51.43 0.49 52.04 0.42 

Not eligible 47.56 0.50 46.51 0.45 

Information not shown 1.02 0.21 1.45 0.27 
English language learner 

Yes 6.03 0.19 6.70 0.15 

No 93.97 0.19 93.26 0.15 
Missing 0.00 0.00 0.04 0.01 
Individualized Education Program 

Yes 11.88 0.12 13.43 0.10 

No 88.12 0.12 86.53 0.10 
Missing 0.00 0.00 0.04 0.02 

+ Not applicable. 


NOTE: NSLP=National School Lunch Program. Detail may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 


(NAEP), 2015 Grade 8 Reading Assessment. 


> The analyses included student-level and school-level demographics comparisons between the analytic sample and 
the full NAEP sample. In addition, NAEP reading performance was compared between the analytic sample and the 


full NAEP sample. 
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Table 3. Reading achievement comparison of the final analytic sample with the NAEP public 
school sample on the 2015 NAEP grade 8 assessment, by operational population 
student subgroups 


Final analytic sample NAEP public school sample 
Average Average 
composite scale | Standard | composite scale | Standard 

Student demographic characteristics score (weighted) error | score (weighted) error 
Overall score 264.65 0.28 263.99 0.24 
Gender 
Male 259.94 0.29 259.10 0.26 
Female 269.49 0.38 269.05 0.32 
Race/ethnicity 
White 273.08 0.24 273.12 0.23 
Black 248.08 0.52 247.17 0.47 
Hispanic 253.44 0.53 252.53 0.43 
Asian/Pacific Islander 279.26 1.23 278.76 1.34 
American Indian/Alaska Native 255.43 1.99 252.82 1.70 
Other 267.56 1.14 267.07 1.03 
NSLP eligibility 

Eligible 253.54 0.33 252.55 0.29 

Not eligible 276.58 0.33 276.36 0.32 

Information not shown 268.52 3.89 273.23 2.97 
English language learner 

Yes 224.36 1.01 222.54 0.90 

No 267.23 0.27 266.73 0.24 

Missing t t 257.77 9.22 
Individualized Education Program 

Yes 230.42 0.69 229.12 0.60 

No 269.26 0.29 268.82 0.25 

Missing + + 254.81 10.14 


+ Not applicable. 

NOTE: NSLP=National School Lunch Program. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 


Because the HLM analysis includes the school-level, the representativeness of schools in the 
final analytic sample is also important for the generalization of the study findings to the full 
NAEP public school sample. Table 4 compares school demographic characteristics between the 
final analytic sample and the original NAEP public school sample. The results show that the final 
analytic sample was also very similar to the original NAEP public sample in terms of school- 
level basic demographic characteristics. The distributions of school key demographic variables 
included in the study were almost the same between the final analytic sample and the NAEP 
public school sample. For example, 14.5 percent of schools had 0—25 percent of students eligible 
for NSLP in the final analytic sample, and the percentage for the NAEP public school sample 
was 14.6. Similar patterns were observed for other variables. 
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Table 4. Comparison of the final analytic sample with the 2015 grade 8 public school population, by 
school characteristics 


NAEP public school 

Final analytic sample population! 

Weighted Standard Weighted Standard 
School characteristics Percentage error Percentage error 
Percent of students eligible for NSLP 
0-25 percent 14.51 0.66 14.57 0.67 
26-50 percent 28.32 0.82 28.51 0.82 
51-75 percent 28.03 1.21 27.56 1.18 
76-100 percent 29.15 0.99 29.35 0.99 
Percent enrollment identified as English language learner 
0-25 percent 92.44 0.71 92.51 0.66 
26-50 percent 5.09 0.60 5.03 0.54 
51-75 percent 2.01 0.51 1.86 0.45 
76-100 percent 0.45 0.12 0.59 0.12 
Percent of students in special education 
0-25 percent 91.85 0.59 91.51 0.56 
26-50 percent 6.98 0.62 7.03 0.57 
51-75 percent 0.27 0.06 0.34 0.08 
76-100 percent 0.91 0.25 1.12 0.28 
Percent of students absent on average day 
0-5 percent 80.99 0.83 80.92 0.74 
More than 5 percent 19.01 0.83 19.08 0.74 
School percent of Black students (mean) 15.30 0.50 16.27 0.39 
School percent of Hispanic students (mean) 19.65 0.48 19.78 0.34 


' Statistics for the NAEP public school population exclude schools with missing data. The average missing rate for variables 
presented in this table is around 9 percent, except the variable "percent of students eligible for NSLP" which had a missing rate of 
17 percent. Additional sensitive analysis has been conducted to check whether there were any potential biases for this variable. 
No bias was detected based on the analysis results. 

NOTE: NSLP=National School Lunch Program. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 


Overall, student and school characteristics between the final analytic sample and NAEP public 
school sample were very similar. For the purposes of this study, the final analytic sample can be 
taken as representative of the entire NAEP public school sample. 


Variables used in the HLM analysis 


The selection of variables used in the HLM models was based on the theoretical background 
discussed in the previous section, variables which demonstrated an important relationship 
between them and reading achievement. 
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Student-level variables 
Demographic background variables 


Student background variables included gender, race/ethnicity, socioeconomic status (SES), ELL 
status, and IEP status. Gender and race/ethnicity were dummy coded variables. Female students 
were coded as | and the reference group was male students. Race/ethnicity categories were 
coded as dummy variables for Black, Hispanic, Asian/Pacific Islander, and Other’, with White as 
the reference group. SES was coded using a composite developed in previous research by Xie 
and Broer in 2016. The resulting index, which incorporates NSLP eligibility, parental education, 
and other items from the NAEP contextual questionnaire, takes integer values ranging from 0 to 
16 (see Appendix A for details). Student ELL and IEP status were also dummy coded 
variables—students who were ELL and had IEPs were both coded as 1, those not, 0. 


Reading motivation variables 


Reading motivation is defined as individuals’ beliefs about reading (Guthrie and Wigfield 2000), 
including both intrinsic and extrinsic factors. The current study includes five items drawn from 
the NAEP student questionnaire related to reading motivation. The selection of these five items 
was based on the theories of reading motivation discussed in the previous section. These five 
items were: “When I read books, I learn a lot” (utility-extrinsic motivation); “Reading is one of 
my favorite activities” (enjoyment/interest-intrinsic motivation); “Reading is enjoyable” 
(enjoyment/interest-intrinsic motivation); “Read for fun on your own time” (enjoyment/interest- 
intrinsic motivation”’); “Talk with your friends or family about something you have read.” (proc- 
social goal-extrinsic motivation). The first three items were a 4-point Likert scale consisting of 
“strongly disagree,” “disagree,” “agree,” and “strongly agree.” The fourth and fifth items also 
contains four response categories: never or hardly ever, once or twice a month, once or twice a 
week, and almost every day. A value of 0 was assigned to the least positive response, and a value 
of 3 was assigned to the most positive response. 


Exploratory and confirmatory factor analyses for reading motivation 


To examine the underlying structure of the five student-reported variables related to reading 
motivation, both exploratory and confirmatory factory analyses (EFA* and CFA) were 
conducted. EFA results suggested that a one-factor model fit the data best, which was confirmed 
by the CFA results. Model fit indices from the CFA indicated that the hypothesized one-factor 
model fit the data well (Hu and Bentler 1999): the Root Mean Square Error of Approximation 
(RMSEA) was 0.06, the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI) were 
both 0.99. Standardized factor loadings of the five items ranged from 0.63 to 0.93, indicating that 
all five items moderately to strongly relate to the latent construct which was defined as reading 
motivation (See Table 5). The one-factor structure confirmed by the EFA and CFA analyses 


3 Other includes Native American/Alaska Natives and students who are unclassified. 

4 Results from exploratory factor analysis (EFA) with one-factor solution are not reported in the table as they are 
same as the confirmatory factor analysis model reported in Table 5. EFA results with greater number of factors were 
not reported as they did not converge. 
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indicated that while intrinsic and extrinsic motivation may be conceptually distinct, in fact they 
are highly correlated with each other and need to be studied together (Schiefel et al. 2012). 


Table 5. Confirmatory factor analysis of NAEP reading motivation items 


Model fit indices 
Root Mean Square Error of 0.06 
Approximation (RMSEA) 
Comparative Fit Index (CFD 0.99 
Tucker-Lewis Index (TLD 0.99 
Variable Description of variables Standardized factor 
loading 
R833001! Learn a lot when reading books 0.63 
R833101! Reading is a favorite activity 0.93 
R833002! Reading is enjoyable 0.89 
R8334017 How often read for fun on own 0.82 
R833501? How often talk with friends about what you read 0.61 


' Four response categories ranging from “strongly disagree” with the statement to “strongly agree.” 

> Four response categories ranging from “never or hardly ever” to “almost every day.” 

NOTE: All variables were recoded from 0 to 3. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 


Given that the adequate model fit and the factor loadings of five items from the one-factor CFA 
results, a reading motivation index was constructed by averaging students’ responses on the five 
reading motivation questionnaire items. Internal consistency reliability (Cronbach 1951) for the 
reading motivation index was 0.84. 


School-level variables 


School characteristics included the following variables: percentage of students eligible for the 
NSLP; percentage of students identified as ELL; percentage of students with IEPs; percentage of 
Black students; and percentage of Hispanic students. For percentage of students eligible for the 
NSLP and percentage of students identified as IEP, the original categories were: 0 percent, 1—5 
percent, 6—10 percent, 11—25 percent, 26—50 percent, 51—75 percent, 76—90 percent, and 91—100 
percent. To create equal intervals for facilitating the interpretation of HLM results, these two 
variables were recoded into quartile categories. Percentage of Black students and percentage of 
Hispanic students were continuous variables ranging from 0 to 100 percent. 


In addition to the school-level demographic variables listed above, the study also accounted for 
school-level academic environment as captured by the following two variables: student-level 
aggregate of reading motivation and the percentage of students absent on an average day. The 
percentage of students absent on an average day is a categorical variable® with two categories: 
0-5 percent and more than 5 percent. 


6 The original categories for this variable were 0—2 percent, 3-5 percent, 6-10 percent, and more than 10 percent. It was 
impossible to create an equal interval variable; therefore, two categories were used. 
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Outcome variables 
2015 Reading Grade 8 NAEP Assessment scores 


NAEP reports an overall score for reading, which is a composite of two subscales: information 
texts and literary texts. The overall score of the 2015 grade 8 NAEP reading performance (scale 
scores) was the outcome variable used in the current study. NAEP scale individual level scores 
are reported as “plausible values” because by design, students are administered only a small 
subset of the total pool of assessment items, not the entire assessment.° Multiple imputation 
procedures are used to produce a set of twenty “plausible values” (e.g., plausible scores) for each 
student taking the reading assessment. In generating the plausible values, NAEP uses a 
“conditioning model” that includes all the variables from the various contextual questionnaires 
NAEP collects along with the responses of students to the particular portion of the assessment 
items that they are assigned. 


Analysis 
Hierarchical Linear Modeling 


The primary purpose of this study is to (a) examine the relationship between reading motivation 
and reading achievement and (b) characterize the relationship by student- and school-level 
characteristics, the latter of which requires modeling a nested data structure (i.e., students nested 
within schools) in order to estimate within- and between-cluster relationships. 


A series of hierarchical linear models (HLM) were used to address the research questions. HLM 
allows for the characteristics of both students and schools to be explicitly modeled, each with its 
own main effects, covariances, and variance structures. The HLM analysis used a two-level 
HLM for reading achievement. Level 1 consisted of the student-level variables: demographics 
and reading motivation. Level 2 included school-level demographic variables and aggregated 
school-level mean reading motivation. 


A series of HLM models was developed starting with a null model to a set of contextual models 
at both the individual and school levels. First, a fully unconditional model without any predictors 
at both levels was estimated to determine how much of the variance in the dependent variable 
(NAEP grade 8 reading scores) could be accounted for by school-level differences. A substantial 
variance between schools would justify the need to use HLM analysis. Then, assuming a HLM 
model is needed, six sequential HLM models will be conducted to address the six research 
questions delineated in the previous section and shown again below. 


Research question 1: Is reading motivation significantly associated with NAEP reading scores? 
To address this question, model | (a within-school model [level 1]) with only one predictor (1.e., 


students’ reading motivation) at the student level was estimated. Model | is unconditional at 
level 2 (between school), which means no school-level variables are included to model the 


6 For more information, see NAEP technical documentation on the web: http://nces.ed.gov/nationsreportcard/tdw/analysis/ 
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dependent variable. This model is to examine whether there is a significant association between 
reading motivation and reading achievement without controlling for any confounding variables. 


Research question 2: Does the association between motivation and achievement (investigated in 
research question 1) persist even after student social demographic characteristics are taken into 
account? 


Building upon model 1, student social demographic variables were entered into model 2, 
including the following dummy variables: gender (male=0 and female=1), Black student 
(White=0 and Black=1), Hispanic student (White=0 and Hispanic=1), Asian/Pacific Islander 
student (White=0 and Asian/Pacific Islander =1), Other student (White=0 and Other=1), 
eligibility for the National School Lunch Program (NSLP) (no=0 and yes=1), English language 
learner (ELL) status (no=0 and yes=1), and Individualized Education Program (IEP) status (no=0 
and yes=1). This model was to test whether the relationship between reading motivation and 
reading achievement detected in model 1 changes after taking the social demographic variables 
into account. This model is also unconditional at level 2 (between school), which means no 
school-level variables are included to model the dependent variable. 


Research question 3: Does the association investigated in research question 2 between 
motivation and achievement vary by gender, race/ethnicity, socioeconomic status, English 
language learner status, and individualized education program status? In addition, does the 
association investigated in research question 2 between motivation and achievement vary across 
schools? Do student level characteristics including motivation and demographics help to explain 
White-Black, White-Hispanic and ELL-non ELL students’ achievement gaps? 


Building on model 2, model 3 tests whether the relationship between reading motivation and 
reading achievement identified in model 2 varies by students’ demographic variables. To 
examine this question, eight interaction terms were entered in model 2: Female <x Motivation, 
Black x Motivation, Hispanic x Motivation, Asian x Motivation, Others x Motivation, 
Socioeconomic status (SES) x Motivation, ELL x Motivation, and IEP x Motivation. 


To examine whether the effect of reading motivation on reading achievement varies across 
schools, a motivation random slope effect parameter was also entered into model 3. Ifa 
significant random effect of motivation is observed for this model, subsequent HLM models 
would need to include a term to capture the reading motivation random slope effect. However, a 
fixed-effect model would be used if the effect of reading motivation is found to be not 
significantly different across schools. 


Research question 4: Does school-level mean motivation explain school-level variations in 
student reading achievement across schools? 


Building upon model 3, model 4 extends the HLM model from within-school level to the 
between-school level (from student level to school level). For model 4, a single school-level 
predictor (aggregated school-level mean reading motivation) was added to test whether there is a 
significant association between school-level mean motivation and student reading achievement. 


Research question 5: Do school-level characteristics (social demographics) explain school-level 
variations in student reading achievement across schools? 
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Building upon model 4, school socio-demographic variables were entered into model 5, 
including proportion Black, proportion Hispanic, proportion ELL, proportion with IEPs, 
proportion eligible for NSLP, proportion absent, and aggregated school-level mean SES. 


Research question 6: Do school-level characteristics (social demographics) explain the 
associations between reading motivation and reading achievement across schools? 


If a significant reading motivation random effect is observed in model 3, model 6 will be 
developed to test whether the random effects of reading motivation on reading achievement can 
be explained by school-level characteristics. However, if no significant reading motivation 
random effects are identified, the analysis will stop at model 5. 


The final two-level conventional model (Raudenbush and Bryk 2002) is expressed as follows. 
The student-level equation is: 


¥;; = Bo; + Bi;(MOT);; + B.;(Female);; + B3;(Black);; + By; (Hispanic) ;; + 
Bs ;(Asian);; + Bgj(Others);; + B7;(SES)j; + Bgj(ELL)i; + Boj UEP); + 
Bio;(Female x MOT);; + B,,;(Black x MOT);; + B,2;(Hispanic x MOT);; + 
B3;(Asian X MOT);; + By4;(Others x MOT);; + Bis ;(SES x MOT);; + 
Bi6j(ELL x MOT);; + By7;(IEP x MOT);; + 7; 


where 


‘J is the reading achievement of grade 8 student i in school j (j = 1, ..., m schools); 


Boj is the adjusted mean reading achievement in school j after controlling for all 
predictors; 


Bi; 


Bis: 
through “7 are level-1 coefficients; and 


. is the level-1 random effect or residual. 


The school-level equation is: 


Boj = Yoo + Yo1 MEAN MOT); + ¥o2(PROP_Black) ; + ¥o3(PROP_Hispanic) ; + 
Yoa(PROP_ELL) ; + Yos(PROP_IEP); + Yog(PROP_NSLP) ; + ¥o7(PROP_ABS) ; + 
Yos( MEAN SES); + Uoj; 


Bij = Y10> 
Bo; = ¥20> 


Also, Pi = ¥r0 (y =3, 4, ...8) 
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where 


Yoo . ‘ : 
is the average of the school means on reading achievement across schools; 


You through 18 are level-2 coefficients. For instance, Y01 is the main effect of mean 
motivation on Poj, 


Combining the school-level and student-level models yields:’ 


Yi; = Yoo + Yo.( MEAN MOT); + Yo2(PROPpiack); + Yo3(PROPrispanic) , + Yo4(PROP311) 


+ Yos(PROPrep) ; + Yoo(PROPysip); + Yo7(PROPags); + Yogp(MEAN SES) ; 

+ ¥i19(MOT);; + Y20(Female);; + ¥39(Black);; + Y49(Hispanic);; 

+ ¥50(Asian);; + Ye0(Others);; + ¥70(SES)j; + Ygo(ELL)i; + Yoo UEP); 

+ V110(Female x MOT);; + Y129(Black x MOT);; + ¥139(Hispanic x MOT);; 
+ ¥149(Asian X MOT) ;; + Y¥150(Others X MOT);; + Yig9(SES x MOT); ; 

+ Y170(ELL x MOT);; + Vigo JEP x MOT);; + Uoj + Vij 


Centering 


Centering decisions are important in conducting HLM analysis, which have substantial 
implications for the interpretation of the parameter estimates. There are generally two main 
centering techniques for independent variables in HLM analysis, namely grand mean centering 
and group mean centering (McCoach 2010). Grand mean centering subtracts from an 
individual’s observed value for a variable the mean of that variable taken over the full sample. 
Group mean centering subtracts the mean taken over the individual's group. For the current 
study, group mean refers to the mean of the student-level variables for a school that an individual 
student belongs to. The grand mean centered scores contain both within- and between-group 
variation, containing the mixture of the level 1 and level 2 associations between the dependent 
and independent variables. The group mean centered scores are uncorrelated with level 2 
variables, which have regression coefficients presenting a pure estimate of the level 1 
relationship between the dependent and independent variables (Enders and Tofighi 2007; 
Raudenbush and Bryk 2002). As such, the transformed score captures a student’s standing 
relative to other students in the group. 


For the current study, group-mean centering was applied for all the student-level variables. In 
group-mean centering these key variables, all the between-school variations were removed from 
these predictors and thus produced accurate and unbiased regression coefficient estimates 
(Enders and Tofighi 2007; Raudenbush and Bryk 2002). However, grand-mean centering was 
used for all the school-level variables. 


Weighting 


NAEP uses stratified sampling at the school level and random sampling at the student level for 
each school. “Because of the unequal school and therefore joint selection probabilities in NAEP, 


T Note that for simplicity’s sake, we did not show centering in the equations. 
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the sample must be weighted during analysis so that it will properly reflect the population 
characteristics” (Beaton et al. 2011, p. 147). Previous research developed methods of correct 
sample weighting for estimation of HLMs. One method described by Pfeffermann et al. (1998) 
works well for NAEP. This method requires the use of school weights and student conditional 
weights. The student conditional weights are constructed by dividing the student joint weights by 
the school weights. Also, as shown in Pfeffermann et al. (1998), when the sampling is random, 
scaling the student conditional weights helps to reduce bias in estimates of the variance 
components. 


Homogeneity assumptions in HLM 


Two homogeneity assumptions regarding level-1 variances were tested. One is whether the 
residual variances of the outcome (NAEP reading plausible values) are the same among schools. 
This assumption was checked by the scatterplot of individual school’s residual variances of the 
outcome. As seen from Appendix B, the residual variances distributed evenly, which indicate 
that the hypothesis that the residual variances of NAEP reading plausible values are the same 
among schools could not be rejected. The other assumption is that the level-1 predictors are 
independent of level-1 residuals. This assumption was checked by plotting level-1 residuals 
against the fitted values of NAEP reading plausible values. Appendix C presents the plot of 
level-1 residuals against fitted values of NAEP reading plausible values. A minor negative 
relationship was observed between level-1 residuals and predicted values of NAEP reading 
plausible values indicating that schools with higher predicted mean NAEP plausible values tend 
to have slightly smaller within-school residual variances. This could result in some impact on 
the HLM model results. However, all student-level predictors in this study were centered at 
school level and variables added at the school level would not affect slope estimates at the 
student level. Therefore, this issue might not affect the results for the main research question 
predictor variables, which are student-level. 


MPLUS (8.0 version) was used for HLM analysis, which implements the method of adjusting 
student sample weights. Student- and school-level weights were used in the analysis. 


Results 


Descriptive statistics for NAEP reading achievement and reading motivation 


The descriptive statistics presented in Table 6 provide a general summary of students’ reading 
achievement and achievement gaps, as measured by the 2015 Grade 8 National Assessment of 
Educational Progress (NAEP) reading assessment. While NAEP grade 8 reading in theory has a 
range of 0 to 500, in fact the observed range for the current study is around 390 points, ranging 
from around 30 to 420. The mean and standard deviation of the 2015 reading assessment in this 
sample were 265 and 34.31, respectively. As seen in Table 6 the average scale scores for male 
and female students were 260 and 269, respectively—a statistically significant difference. In 
other words, there was a male-female achievement gap in NAEP reading scores. There were also 
achievement gaps across racial/ethnic groups. The average scale scores for White, Black, and 
Hispanic students were 273, 248, and 253, respectively. That is, there was a 25-point gap 
between White and Black students (0.74 standard deviations), and a 20-point gap between White 
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and Hispanic students (0.59 standard deviations) (Table 6). Both the White-Black and White- 
Hispanic gaps were statistically significant. Also, as seen in Table 6 students who were eligible 
for the National School Lunch Program (NSLP) had significantly lower scores compared to 
those who were not eligible for the NSLP (254 versus 277). Similarly, students who were 
English language learners (ELL) and who had Individualized Education Programs (IEPs) scored 
significantly lower (224 and 230, respectively) compared to students who were not ELL (267) 
and did not have IEPs (269). 


Table 6. Reading achievement by student demographic variables 


Final analytic sample 
Average scale score 

Student demographic characteristics (weighted) Standard error 
Overall score 264.65 0.28 
Gender 
Male (reference group) 259.94 0.29 
Female 269.49* 0.38 
Race/ethnicity 
White (reference group) 273.08 0.24 
Black 248.08* 0.52 
Hispanic 253.44* 0.53 
Asian/Pacific Islander 279.26* 1.23 
Other 264.18* 1.06 
NSLP eligibility 

Eligible (reference group) 253.54 0.33 

Not eligible 276.58* 0.33 
English language learner 

Yes (reference group) 224.36 1.01 

No 267.23* 0.27 
Individualized Education Program 

Yes (reference group) 230.42 0.69 

No 269.26* 0.29 


* Indicates the mean of the subgroup was significantly different from that of the reference group, at p < .05. 

NOTE: NSLP=National School Lunch Program. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 


Descriptive statistics for student reading motivation are presented in Table 7. The overall mean 
for reading motivation was 1.43. Female students reported significantly higher reading 
motivation scores (mean=1.58) than male students (mean=1.28). White students (mean=1.44) 
scored significantly higher on reading motivation than Hispanic students (mean=1.38). In 
contrast, White and Black students (mean=1.43) were not significantly different on reading 
motivation. However, Asian/Pacific Islander students (mean=1.62) scored significantly higher on 
reading motivation than White students. In addition, students who were not eligible for the NSLP 
had significantly higher scores on reading motivation than students who were eligible for the 
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Table 7. Reading motivation by student demographic variables 


Final analytic sample 

Student demographic characteristics Average Standard Error 
Overall score 1.43 0.00 
Gender 
Male (reference group) 1.28 0.00 
Female 1.58* 0.01 
Race/ethnicity 
White (reference group) 1.44 0.00 
Black 1.43 0.01 
Hispanic 1.38* 0.01 
Asian/Pacific Islander 1.62* 0.02 
Other 1.41 0.02 
NSLP eligibility 

Eligible (reference group) 1.39 0.01 

Not eligible 1.47* 0.00 
English language learner 

Yes (reference group) 1.32 0.02 

No 1.44* 0.00 
Individualized Education Program 

Yes (reference group) 1.29 0.01 

No 1.45* 0.00 


* Indicates the mean of the subgroup was significantly different from that of the reference group, at p < .05. 

NOTE: NSLP=National School Lunch Program. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 


HLM results 


Simple correlation analyses among dependent and independent variables included in the 
hierarchal linear models (HLM) were conducted before HLM models were developed. The 
detailed correlations are displayed in Appendices D and E. Overall, student-level reading 
motivation and school-level mean motivation were moderately correlated with grade 8 NAEP 
reading scores (r=0.34 and 0.40, respectively). 


A series of hierarchical linear models was then developed to examine the association between 
motivation and reading achievement. Table 8 presents the HLM results. 


Model 0, the HLM null model, provides a baseline for the decomposition of variance. The total 
variance is the sum of the two displayed components: 1178.07 = 215.38 (school level) + 962.69 
(student-level). The intraclass correlation (ICC), or the proportion of variance in student reading 
achievement between schools is as follows: ICC = between-school variance (t) /total variance (t 
+ 67) = 215.38/ (215.38 + 962.69) = 0.18, which means that 18 percent of the total variance in 8" 
grade reading achievement is attributable to between-school heterogeneity, and 82 percent is 
attributable to within-school heterogeneity. This substantial variation across schools justified the 
need to use HLM for further analysis. Hedges and Hedberg (2007) suggested that ICC value of 
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0.20 is plausible for school-based clustering. This model also indicates that the school reading 
achievement mean for all schools was 262.78 points. 


The subsequent section presents the HLM results to address the six research questions. 
Research question 1: Is reading motivation significantly associated with NAEP reading scores? 


Model 1, which adds the student motivation variable, indicates that students’ reading motivation 
is significantly associated with their reading achievement. Without controlling for any 
demographic or other related variables, a one-unit difference in student motivation (rated on a 4- 
point scale) results in an estimated 15.11-point difference in reading achievement, which is 
roughly half of the standard deviation (34.31%). The addition of the student motivation variable 
accounts for 12 percent of the within- school variance.’ 


8 This is the standard deviation for the final analytic sample. The standard deviation for grade 8 reading by all students from 
national public schools is 35 points. 

° Note that the change in between-school variance is not reported for this model (and Models 2 and 3) because only student-level 
variables were added. 
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Table 8. Summary of results from hierarchical linear models (HLM) 


Model 2: 
Model 0: Model 1: Student Motivation + Model 3: Student Motivation + 
Null Model Student Motivation Demographics Demographics + Interactions 
Variable Est. SE Est. SE Est. SE Est. SE 
Fixed effects 
Intercept: school mean 
Achievement 262.78""" 0.42 262.76°"* 0.42 262.71°"* 0.43 262.71°"" 0.43 
Level 1 (student level) 
Motivation 15.11" 0.23 11.97" 0.22 10.67°*" 0.80 
Female 3.14°" 0.32 2.29""" 0.64 
Black -13.26""" 0.57 -11.47"" 1.11 
Hispanic -2.23""* 0.58 -0.41 1.03 
Asian 5.28" 1.01 7.53°" 1.98 
Other -2.69"*" 0.85 -2.09 1.93 
SES 1.83°"" 0.05 1.60°"* 0.09 
ELL -28.00°** 1.17 -19.98"** 2.23 
IEP -31.66""" 0.62 -31.64""" 1.19 
Female <x Motivation 0.58 0.41 
Black x Motivation -1.22 0.65 
Hispanic x Motivation -1.33" 0.60 
Asian x Motivation -1.36 1.06 
Other x Motivation -0.40 1.14 
SES x Motivation 0.16" 0.06 
ELL x Motivation -6.08""" 1.43 
IEP x Motivation -0.04 0.84 
Random effects 
Intercept (variance 
between schools) 215.38""" | 10.28 223.15" 10.73 241.52°"* 12.39 241.85°"" 12.43 
Level | (variance 
within schools) 962.69"** 10.49 847.32" 10.07 608.72°"* 5.91 606.98""* 5.89 
Intraclass correlation 0.18 0.21 0.28 0.28 
Variance in achievement 
within schools explained (%) | Tt 12 37 37 


*** D < .001; ** p< .01; *p<.05 


23 


Model 4: School Motivation Model 5: School Motivation + School-Level Demographics 
Variable Est. SE Est. SE 
Fixed effects 
Intercept: school mean achievement 262.81°"" 0.40 277.32" 2.71 
Level 1 (student level) 
Motivation 10.67°"* 0.80 10.67°** 0.80 
Female 2.29°"* 0.64 2.29" 0.64 
Black -11.47"™" 1.11 -11.47™" 1.11 
Hispanic -0.41 1.03 -0.41 1.03 
Asian 753°" 1.98 7.53" 1.98 
Other -2.09 1.93 -2.09 1.93 
SES 1.60°"* 0.09 1.60" 0.09 
ELL -19.98""* 2.23 -19.98""* 2.23 
TEP -31.64°™" 1.19 -31.64°"* 1.19 
Female x Motivation 0.58 0.41 0.58 0.41 
Black x Motivation -1.22 0.65 -1.22 0.65 
Hispanic x Motivation -1,33" 0.60 -1,33" 0.60 
Asian x Motivation -1.37 1.06 -1.37 1.06 
Others x Motivation -0.40 1.14 -0.40 1.14 
SES x Motivation 0.16"** 0.06 0.16"** 0.06 
ELL x Motivation -6.08°"* 1.43 -6.08"" 1.43 
IEP x Motivation -0.04 0.83 -0.04 0.83 
Level 2 (school level) 
Mean motivation BOT” 2.56 16.98" 1.81 
Proportion Black -0.14"" 0.01 
Proportion Hispanic -0.02 0.02 
Proportion ELL -0.18 0.34 
Proportion IEP -1.23" 0.44 
Proportion NSLP -0.89°""" 0.27 
Proportion absent -0.66 0.42 
Mean SES 3.32" 0.25 
Random effects 
Intercept (variance between schools) 210.39°"" 9.88 65.57" 4.71 
Level | (variance within schools) 606.70" 5.92 608.23" 5.95 
Intraclass correlation 0.26 0.10 
R-squared achievement btw schools (%) 13 73 


*** 1D < 001; ** p< .01; * p<.05. ¢ Not applicable. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Grade 8 Reading Assessment. 


Research question 2: Does the association between motivation and achievement persist even 
after student social demographic characteristics are accounted for? 


Model 2 added student-level demographic variables, including seven dummy variables (Female, 
Black, Hispanic, Asian, Other,'° ELL, and IEP) and one composite score for students’ 
socioeconomic status (SES), which ranged from 0 to 16. The results show that when differences 
in individual students’ background were accounted for, the association between student 
motivation and reading achievement remained statistically significant. Specifically, when 
controlling for all student-level demographic variables (1.e., gender, race/ethnicity, SES, ELL, 
and IEP), each unit difference in student motivation is estimated to result in a difference of about 
12 points in reading achievement, 35 percent of the standard deviation. All student-level 
demographic variables were statistically significantly associated with reading achievement. 
Among these variables, the estimated coefficients for Black, SES, ELL, and IEP are markedly 
large. Specifically, if the student was Black instead of White, the achievement would be an 
estimated 13.26 points lower within the same school (given that no interaction effects were 
found). This gap was 2.23 points for Hispanic students and 2.69 points for Other students. By 
contrast, Asian students scored 5.28 points higher than their White counterparts. ELL students 
scored 28 points lower than students within the same school who were not classified as ELL, and 
this difference was almost 32 points for students with IEPs. Note that both the ELLs and 
students with IEPs score nearly one full standard deviation lower than their counterparts. The 
results show that students’ family background, as measured by SES, is also a powerful predictor 
of reading achievement. The estimated coefficient associated with SES, which was measured on 
a scale ranging from 0 to 16, is 1.83. This means, compared to their counterparts with the highest 
SES, students with the lowest SES score 29.28 points lower on the NAEP reading assessment, 
which is almost one standard deviation, after taking into account all other student-level variables 
(given that no interaction effects were found). Overall, student-level demographics explained an 
additional 25 percent of the variance within schools (37 percent explained in model 2 compared 
to 12 percent explained in Model 1). 


Research question 3: Does the association investigated in research question 2 between 
motivation and achievement vary by gender, race/ethnicity, socioeconomic status, English 
language learner status, and individualized education program status? In addition, does the 
association investigated in research question 2 between motivation and achievement vary across 
schools? Do student level characteristics including motivation and demographics help to explain 
White-Black, White-Hispanic and ELL-non ELL students’ achievement gaps? 


Model 3 added eight interaction terms between student motivation and student-level 
demographic variables to determine whether the association between motivation and 
achievement varied by demographic variables. The addition of the interaction terms alters the 
interpretation of the main motivation coefficient. In Model 1, the coefficient is interpretable as 
the mean within-school relationship between motivation and achievement across all students. 
While in Model 3, it is interpretable as the mean relationship for the “average” student in each 
school (1.e., students whose background characteristics are set to zero, after background 
characteristics have been centered about their school-level means). The interaction effects are 
then estimates of the difference in motivation slopes associated with an increment of | point in 


‘0 A combination of the American Indian/Alaska Native and Other racial/ethnic categories. 
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the interacted variable. For instance, the coefficient of 0.16 associated with the SES x 
Motivation interaction indicates that students with an SES value 1 point higher than others are 
estimated to have a motivation coefficient 0.16 points higher, all else being equal. For dummy- 
coded indicator variables, this is interpretable as the difference in motivation slopes between the 
indicated group and the reference group. For example, the coefficient of 0.58 for 

Female X Motivation implies that the motivation coefficient for female students is estimated to 
be 0.58 points higher than for male students. 


Model 3’s results indicate that the association between motivation and achievement remained 
statistically significant after the addition of the interaction terms, with a minor decrease in the 
unstandardized coefficients for motivation (i.e., from 11.97 to 10.67). However, significant 
interaction effects were found between motivation and three of the demographic variables: SES 
as well as Hispanic and ELL status. The results for SES indicate that the association between 
motivation and achievement is stronger for students of higher SES—0.16 points per point on the 
SES index. Given the SES index range of 16, this indicates that the slope on motivation is 
estimated to be 2.56 points higher on the dependent variable for the highest SES students than 
for the lowest taking into account all other variables in the model. For Hispanic and ELL 
students, the interaction effects indicate that the relationship between motivation and 
achievement is significantly weaker for Hispanic students than for White students (-1.33 points), 
and far weaker for ELL students than non-ELL students (-6.08), everything else being equal. 


The results also suggest that the Hispanic-White achievement gap in reading is moderated by 
motivation. Specifically, the non-significant coefficient on the Hispanic main effect (-0.41) 
reveals that for students whose motivation is equal to their school-level average and are identical 
on other predictors, there is no statistically significant gap between Hispanic and White students; 
at other levels of motivation, however, a significant gap may exist. To illustrate the interaction 
effect of being Hispanic students and motivation, we used the results from Model 5 to produce 
expressions for the difference between average scores for Hispanic and White students who are 
identical on the remaining variables in appendix F. 


The complicated relationships among student reading motivation, being Hispanic, being ELL 
and NAEP reading performance (results from Model 5) are graphed in Figure 1. Four 
observations can be noted from the figure. (1) Regardless of students’ race/ethnicity and ELL 
status, reading motivation is positively associated with NAEP reading score. This suggests that 
higher motivation is beneficial to all students’ reading performance. (2) For both ELL and non- 
ELL students, the Hispanic slope was significantly lower than the White slope, but the 
differences in their respective predicted scores are small. This suggests that the effect of 
motivation on reading achievement is partially moderated by whether a student is Hispanic even 
though the achievement gap between Hispanic and White students (NAEP reading score 
difference) is not statistically significant. (3) A substantial interaction effect between motivation 
and student’ ELL status was observed. That is, the higher the motivation, the more pronounced 
the effect was for non-ELL students compared to ELL students. Although a significant 
interaction effect between motivation and being Hispanic or White was observed, the effect was 
small compared to that of being an ELL student or not. (4) The expected NAEP reading score for 
White and Hispanic students regardless their ELL status non-ELL students and ELL students 
was not statistically different, after controlling for students’ reading motivation and social- 
demographic background. This indicates the reading achievement gap between White and 
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Hispanic students is largely due to the differences between ELL and non-ELL students. That 
is, ELL status appears to be the key variable to explain the achievement gap between White 
and Hispanic students. 


Figure 1. Predicated NAEP reading score based on student motivation level, Hispanic 
and ELL status 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2015 Grade 8 Reading Assessment. 


The inclusion of interaction terms did not increase the variance within schools explained when 
compared to Model 2. 


In order to investigate whether the association between motivation and achievement varies across 
schools (i.e., the second part of RQ3), we further examined Model 3 by allowing random slopes 
for the motivation variable. The estimated variance of the slopes is 3.12 (see Appendix G), which 
is not statistically significant. Based on this result, we infer that the association between 
motivation and achievement within schools does not vary significantly across the population of 
schools. Therefore, in the subsequent models (i.e., Models 4 and 5), motivation is treated as a 
fixed effect variable. 


Research question 4: Does school-level mean motivation explain school-level variations in 
student reading achievement across schools? 
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Model 4, which added the school mean motivation variable, indicates that school mean 
motivation is highly associated with school mean reading achievement. The estimated coefficient 
for school mean motivation suggests that for each unit difference in school mean motivation, 
there is an estimated 23.07-point difference in school mean reading achievement, which is 70 
percent of the standard deviation. The inclusion of the school mean motivation variable 
explained 13 percent of the variance between schools, using Model 3 as the baseline. !! This 
suggests that, in addition to student-level motivation and students’ own backgrounds, school- 
level motivation sustainably influences reading achievement. 


Research question 5: Do school-level characteristics (social demographics) explain school-level 
variations in student reading achievement across schools? 


In our final model, Model 5, we included school-level demographic variables. The inclusion of 
the school-level demographic variables explained an additional 60 percent of the variance 
between schools, suggesting that, in addition to student-level motivation, students’ own 
backgrounds and school-level motivation, the demographic composition of schools also greatly 
influences reading achievement. According to Model 5, the proportion of Black and IEP 
students, the percentage of students eligible for the NSLP, and school SES were significantly 
correlated with achievement even after taking into account school-level mean motivational levels 
and the other demographic variables in the model. The differences in magnitude of the 
coefficients for school proportions in comparison to the others (e.g., mean motivation) are the 
results of differences in scaling: the proportion of Black students assume values from 0 to 100; 
variables measuring percentage of students identified as IEP and percentage of students eligible 
for the NSLP were coded into quartile categories; and the value of mean SES ranges from 0 to 
16. Thus, when taking into account all other variables, each unit difference in the proportion of 
Black students in a school results in 0.14 points decrease in school mean achievement—only a 
small difference. Likewise, if the percentage of students with IEPs and those eligible for the 
NSLP differs by one unit (one quartile), the school mean achievement is predicted to decrease by 
1.23 and 0.89, respectively, both small differences. Finally, a one-point difference in mean SES 
scores is predicted to result in a 3.32-point difference in school mean achievement, given that all 
other variables are taken into account. 


Research question 6: Do school-level characteristics (social demographics) explain the 
associations between reading motivation and reading achievement across schools? 


As examined earlier in Model 3, the relationship between motivation and achievement within 
schools does not vary significantly across the population of schools, therefore motivation was 
treated as fixed effect variable. As a result, no additional model was developed to address this 
research question. 


"| According to Raudenbush and Bryk (2002), “Technically speaking, the variance explained in a level-2 parameter, 
such as [the random intercept], is conditional on a fixed level-1 specification. As a result, the proportion reduction in 
variance statistics at level 2 are interpretable only for the same level-1 model. Consequently, we recommend that 
researchers develop their level-1 model first, and then proceed to enter level-2 predictors into the analysis” (p. 150). 


28 


Discussion 


Student reading motivation and school-level mean motivation on reading 
achievement 


The primary purposes of the present study were (1) to identify the unique effects of reading 
motivation on reading achievement and (2) characterize this relationship by student- and school- 
level characteristics. The current study found that reading motivation accounted for 12 percent of 
the student-level variance in NAEP reading scores. Also, the analyses indicated that student 
reading motivation is substantially associated with student reading achievement, even after 
taking into account student gender, race/ethnicity, and SES, ELL, IEP statuses, and interaction 
terms. Each unit difference in student reading motivation is predicted to result in a 10.67-point 
difference in NAEP reading scores, which is approximately one third of a standard deviation. 


The findings from this study comport with the literature on the effects of reading motivation on 
reading achievement. Park (2011) investigated motivation constructs (including reading concepts 
and intrinsic and extrinsic motivation) and found that motivation constructs and their interactions 
account for 17 percent of the student-level variance in reading performance for 4" graders after 
controlling for student demographic variables. Guthrie et al. (2007) reported that reading 
motivation explained between 3 and 22 percent of the variance for 4"" graders’ reading 
comprehension when prior reading performance was controlled for. The current study extends 
the literature on the effects of reading motivation on reading achievement to middle school 
students. Although reading motivation decreases as grade level increases (Gottfried, Fleming, 
and Gottfried 2001; Kelley and Decker 2009), reading motivation continues to play an important 
role in student reading achievement across different grade levels, from elementary to middle 
school. Therefore, student reading motivation should be considered as an important element 
when designing reading instruction and activities at school. Brynes (2003) concluded that 
theories of school reform or practices should include variables that can be influenced by school, 
not factors that educators can do little to impact — factors such as students’ social demographic 
background. Student’s reading motivation is one of the significant factors that educators could 
work to improve. Research has shown that teachers can play an important role in motivating 
students (Goodenow, 1993, Noddings, 2015, Ruddell and Ruddell, 2004, and Wentzel, 1997). 
Following a sample of 248 students from 6™ to 8 grade, Wentzel (1997) found that perceived 
teachers’ pedagogical caring was significantly associated with students’ academic motivation at 
grade 8, even taking into account of students’ current level of psychological well-being, previous 
motivation level and academic performance at grade 6. “Teachers who care were described as 
demonstrating democratic interaction styles, developing expectations for student behavior in 
light of individual differences, modeling a “caring” attitude toward their own work, and 
providing constructive feedback” (Wentzel, 1997, p.411). 


Another important finding of the current study is the unique and substantial effect of school-level 
mean motivation on school reading achievement even in the presence of other school 
demographic variables—variables such as mean school-level SES, proportion of racial/ethnic 
minority students, and proportion of IEP students. The estimated coefficient for school mean 
motivation suggests that for each unit difference in school mean motivation, there is a predicted 
17-point difference in school mean reading achievement which is about half of the standard 
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deviation. School-level mean motivation also explained 13 percent of the reading achievement 
variance between schools. 


This substantial positive effect highlights the significant role schools have on promoting 
students’ academic development. While the current study does not shed light on what might 
account for this strong school effect, many researchers and school reformers have suggested that 
school climate makes a difference in student academic learning and growth (Maslowski 2001; 
Hoy et al. 2006). The major levers seen as important for improving school climate are principals’ 
leadership, teachers’ expectations, and inter-personal relationships within and around schools 
(MacNeil, Prater and Bush 2009). The finding from the current study suggests another area that 
schools can work on, promoting school-level motivation, to improve student academic 
achievement. 


Reading motivation and achievement gaps 


In contemporary schooling, achievement gaps between White students and their Black and 
Hispanic peers have been a consistent concern (Jencks and Phillips, 1998). The present study 
contributes to an understanding of the achievement gaps in reading by identifying key student 
characteristics associated with the White-Hispanic and White-Black achievement gap. In 
addition, the achievement gap between ELL students and non-ELL students has become a new 
focus regarding equity issues in education, because ELL students are the fastest growing 
population in this country (Kena et al., 2015). The current study also investigated the 
achievement gap between ELL and non-ELL students and how the effect of reading motivation 
on reading achievement is impacted by student ELL status. 


White-Hispanic achievement gap 


The original achievement gap between White and Hispanic students was 20 points before taking 
into account other explanatory variables, 0.64 standard deviations. After accounting for student 
reading motivation, student demographic variables, and the interaction terms, the achievement 
gap was just 0.41 points and no longer statistically significant. In essence, there is no statistically 
significant achievement gap between Hispanic and White students, among students whose 
reading motivation is equal to their school-level average and are identical on other predictors. 
Specifically, if Hispanic students have the school-level average reading motivation and share the 
same social demographic background investigated in the current study with White students, 
Hispanic middle school students reading achievement level would be the same as that of White 
students. 


However, the present study found that Hispanic students scored lower on the study’s measure of 
reading motivation than did their White peers. Based on Bronfenbrenner’s (1986) ecological 
model of contextual influences on children’s development, home and school are the two primary 
microsystems for children. How to motivate students in reading and other academic activities at 
home and school becomes the key. Research has consistently shown that parents’ educational 
involvement at home and school like providing a supportive learning environment and 
communicating with teachers, is positively associated with students’ academic learning across 
different race/ethnicity groups (Jeynes, 2003 and McWayne et al., 2004). However, research 
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indicates that Hispanic students have less parental involvement at school and less parent-child 
educational discussions compared to White students (Lee and Bowen, 2010). 


We are unaware of other studies that have identified variables that fully explain the achievement 
gaps between White and Hispanic students. Fryer and Levitt (2004) investigated the 
race/ethnicity test score gaps using the Early Childhood Longitudinal Study kindergarten cohort 
(ECLS-K) data. They found that much but not all of the Hispanic-White difference in test scores 
was accounted for by a small number of children’s social demographic variables including 
children’s age, child’s birth weight, SES, Women, Infant, and Children (WIC) participation, 
mother’s age at first birth, and number of children’s books in the home. However, their study did 
not include any motivation measures in the model and their study focused on younger children. 
Future studies need to examine whether this finding can be replicated for other subjects and 
students at different grade levels. 


In addition, the current study found that the net effect of reading motivation on achievement for 
Hispanic students was significantly weaker than that of White students although the effect was 
small. This weaker relationship between reading motivation and reading scores for Hispanic 
students was also documented by other researchers (Graham 1994; Graham and Taylor 2002; 
Unrau and Schlackman 2006). Guthrie et al. (2009) and Unrau and Schlackman (2006) suggests 
that the key elements of motivation for Hispanic students and other unstudied populations may 
be different from that of White students. The development of motivation questionnaires is 
largely based on majority students, which means that they might have a larger predictive power 
for majority students compared to minority students (Proctor, Daley, Louick, Leider, and 
Gardner, 2014). 


Ell-non-ELL achievement gap 


ELL students have become the fastest growing student population in this country (Kena et al., 
2015), but their academic performance is among the lowest. ELL students have lower academic 
achievement, high dropout rates, and high rates of grade retention (KewalRamani, Gilbertson, 
Fox, and Provasnik, 2007). The current study also found that there are glaring achievement 
disparities between ELL and non-ELL students on the NAEP reading assessment. The original 
achievement gap between ELL and non-ELL students was 43 points, 1.26 standard deviations. 
After controlling for student motivation and social demographic background, ELL students 
scored 20 NAEP points lower compared to non-ELL students, which is 0.60 of the standard 
deviation. Similar achievement gaps between ELL and non-ELL students have been documented 
in the research literature (August, McCardle and Shanahan, 2014; Lesaux, 2006). 


Also, this study investigated interactions among motivation, student ELL status, and being 
Hispanic students. The NAEP reading scores for White and Hispanic students after taking into 
account their ELL status and their socio-demographic backgrounds were not statistically 
different. This indicates the reading achievement gap between White and Hispanic students is 
largely due to the differences between ELL and non- ELL students. the ELL status appears to be 
the key variable to explain the achievement gap between white and Hispanic students. Reardon 
and Galindo (2009) used the Early Childhood Longitudinal Study—Kindergarten Class of 1998— 
1999 (ECLS-K) data to investigate the achievement gap between White and Hispanic students in 
elementary school. They found that Hispanic students with limited English proficiency at the 
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start of kindergarten made more rapid gains in reading and math than did students proficient in 
English, suggesting that limited English proficiency may be a major barrier in Hispanic students 
learning process. That is, it is English language proficiency in the current study that accounts for 
reading performance deficits, not being Hispanic. 


The current study found an interaction effect between ELL status and reading motivation: the 
effect of motivation on reading achievement was substantially weaker for ELL students 
compared to non-ELL students. Students having a higher level of reading motivation on average 
are more likely to do well on a reading achievement assessment than those with lower reading 
motivation. However, for ELL students, the effects of motivation seem muted in the absence of a 
strong grasp of the English language. This finding suggests the important role that English 
proficiency plays in reading achievement. Therefore, helping ELL students develop English 
proficiency should be a high priority. 


White-Black achievement gap 


Variables identified in the HLM model were able to explain a substantial amount of the reading 
achievement gap between White and Black students, but not all of it. The original achievement 
gap between White and Black students was 25 points, 0.74 of the 2015 NAEP grade 8 reading 
standard deviation. After taking into account student reading motivation, student demographic 
variables, and the interaction terms, the achievement gap narrowed to 11 points (0.32 standard 
deviation), which is still statistically significant. Only one study could be located that has 
identified a set of variables that fully explain the White-Black reading achievement gaps in the 
first two years of school. Fryer and Levitt (2004) analyzed the ECLS-K data to model the White- 
Black students test scores gap in reading and mathematics. They eliminated the White-Black test 
score gaps in both subjects by including the following variables in their model: a small set of 
children’s background variables including children’s age, child’s birth weight, SES, Women, 
Infant, and Children (WIC) participation, mother’s age at first birth, and number of children’s 
books in the home. These variables were not available to take into account in the current study 
which may account for why the Black-White achievement gap could not be fully explained. 


Limitations 


A small number of items and a narrow breadth in assessing different dimensions of reading 
motivation were the major limitations of the current study. Nonetheless, this study showed that 
the measure built from these items displayed a significant and substantial relationship to grade 8 
reading achievement. Based on the current study, we recommend that more items be developed 
in future NAEP assessment cycles to more fully capture the various aspects of reading 
motivation, including intrinsic motivation, self-efficacy, valuing of reading, perceived difficulty, 
and pro-social goals. Also, future studies need to not only include broader dimensions of reading 
motivation but also pay more attention to their underlying facets, such as reading self-efficacy, 
intrinsic motivation, and extrinsic motivation. 


Reading motivation is assumed to increase individuals’ achievement by stimulating and 
reinforcing positive behaviors and activities beneficial to gains in achievement (Schunk et al. 
2008; Wigfield and Guthrie 1997). Accordingly, reading motivation should be related to 
particular aspects of reading behaviors such as reading amount and reading time, which are 
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important for the development of reading and comprehension skills. However, the current study 
did not include any variables about student reading time, amount, and strategies, which 
foreclosed an examination of the relationship between reading motivation and these types student 
reading behaviors. Therefore, we suggest that future NAEP assessments develop items to capture 
these types of variables as a way to improve the value of future research on reading achievement. 
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Appendix A. Items in the construction of the SES index and their 


coding 


Items 


NSLP eligibility 


No info on NSLP 


Eligible for NSLP 


Not eligible for NSLP 


Parent education 


Missing or unknown pared 


Did not graduate from HS 


Graduated from HS 


Some college after HS 


Graduated from college 


Books 


Missing info on books 


Few books (0-10) 


Enough to fill one shelf (11-15) 


Enough to fill one bookcase (26—100) 


Enough to fill several bookcases (more than 100 books) 


BLWIN]Re | oO 


Household possessions 


Missing info or one HH possession 


2 HH possessions 


3 HH possessions 


4 HH possessions 


5 HH possessions 


6 HH possessions 


BIWILN Re |olo 


NOTE: NSLP=National School Lunch Program. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 


(NAEP), 2015 Grade 8 Reading Assessment. 


Appendix B: The scatter plot of individual school residuals of 
NAEP reading plausible values 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 
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Appendix C: The plot of level-1 residuals against fitted values of 
NAEP reading plausible values 
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SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 


Appendix D. Correlations among student-level variables 
NAEP reading Motivation Female Black Hispanic __Asian Others SES ELL IEP 


NAEP grade 8 reading achievement 1.00 

Motivation 0.34 1.00 

Female 0.15 0.21 1.00 

Black -0.12 -0.01 0.01 1.00 

Hispanic -0.09 -0.03 0.01 -0.29 1.00 

Asian 0.06 0.04 0.01 -0.10 -0.16 1.00 

Others -0.01 -0.01 0.00 -0.09 -0.10 = -0.05 1.00 

SES 0.34 0.16 0.01 -0.07 -0.24 0.02 -0.01 1.00 

English language learners (ELL) -0.24 -0.03 -0.02 -0.09 0.22 0.05 -0.03 -0.17 1.00 

Students with an IEP -0.40 -0.07 -0.11 0.03 0.00 _—_—-0.04 0.01 -0.17 0.06 1.00 


NOTE: IEP=Individualized Education Program. ELL=English language learners. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Grade 8 Reading Assessment. 
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Appendix E. Correlations among school level variables 


NAEP Mean Proportion Proportion Proportion Proportion Proportion Proportion Mean 
reading motivation Black _ Hispanic ELL TEP lunch absent SES 
NAEP grade 8 reading achievement 1.00 
Mean reading motivation 0.40 1.00 
Proportion Black -0.45 0.05 1.00 
Proportion Hispanic -0.29 -0.03 -0.13 1.00 
Proportion ELL -0.28 -0.02 0.00 0.64 1.00 
Proportion IEP -0.18 -0.09 0.15 -0.17 -0.07 1.00 
Proportion NSLP -0.67 -0.09 0.40 0.34 0.28 0.16 1.00 
Proportion absent -0.19 -0.03 0.16 -0.01 0.04 0.19 0.20 1.00 


Mean SES 0.77 0.16 -0.35 -0.47 -0.39 -0.08 -0.79 -0.16 1.00 


NOTE: IEP=Individualized Education Program. ELL=English language learners. NSLP=National School Lunch Program. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Grade 8 Reading Assessment. 
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Appendix F. An expression example of interpreting the Hispanic- 
White achievement gap by motivation. 


To illustrate the interaction effect of being Hispanic students and motivation, we used the results 
from Model 5 to produce expressions for the difference between average scores for Hispanic and 
White students who are identical on the remaining variables. The estimated score for student 7 
can be written: 


Y; = K + 10.67(MOT;) — 0.41(Hispanic) — 1.33(MOT; x Hispanic) (X) 


where K is some linear combination of the other parameter estimates. 


Plugging values into equation (X), the estimate for White students is then: '? 


[¥;|Hispanic; = 0] = K + 10.67(MOT;) — 0.41(0) — 1.33(MOT; x 0) 
= K + 10.67(MOT;) 


For Hispanic students, the estimate is: 


[¥;|Hispanic; = 1] = K + 10.67(MOT;) — 0.41(1) — 1.33(MOT; x 1) 
= K + (10.67 — 1.33)(MOT,;) — 0.41 


= K + 9.34(MOT,) — 0.41 


The estimated Hispanic-White gap is the difference between these two expressions: 


[¥,| Hispanic; = 1] — |¥;|Hispanic; = 0] 
= K + 9.34(MOT;) — 0.41 — (K + 10.67(MOT;)) 


= —1.33(MOT;) — 0.41 (X) 


bs : MOT; 
That is, in our model, the Hispanic-White gap varies by ; 


Without the interaction effects, the achievement gap between Hispanic and White students would 
be 2.23 points (see Model 2). However, after taking into account the interaction effect (Model 3), 


MOT; 
the Hispanic-White achievement gap varies by ’, that is it is a function of the motivation 
effect. Similarly, the estimated achievement gap between ELL and non-ELL students would be 
6.08(MOT;) + 19.98. 


? The school-level proportion of Hispanic students is cancelled out in the final equation and is omitted for clarity. 
F-1 


Appendix G. Summary of results from hierarchical linear models 
(HLM): Student motivation treated as random effects variable 


Model 3 Model 4 Model 5 
Variable Est. SE Est. SE Est. SE 
Fixed effects 
Level | (student level) 
Intercept 262.71°"* 0.43 262.81" 0.40 277.33" 2.72 
Motivation 10.61°** 0.81 10.65°** 0.80 10.66°** 0.80 
Female 2.28°"* 0.64 2.28°"* 0.64 2.28°"* 0.64 
Black -11.44°™ 1.14 -11.49°™ 1.14 -11.52"" 1.12 
Hispanic -0.39 1.03 -0.41 1.04 -0.43 1.03 
Asian 1a 1.98 7.54" 1.98 ial 1.98 
Others -2.10 1.93 -2.11 1.93 -2.11 1.93 
SES ae! 0.10 159°" 0.09 1.60°" 0.09 
ELL -19.90°"* 2.23 -19.92°"* 2.23 -19.87°"* 2.23 
IEP -31.67"" 1.19 -31.67"" 1.19 -31.65°"* 1.18 
Female x Motivation 0.58 0.41 0.58 0.41 0.58 0.41 
Black x Motivation -1.24 0.67 -1.21 0.68 -1.19 0.66 
Hispanic x Motivation -1.35" 0.60 -1,33" 0.60 -1,32" 0.60 
Asian x Motivation -1.36 1.07 -1.37 1.07 -1.35 1.07 
Others x Motivation -0.39 1.14 -0.38 1.14 -0.39 1.14 
SES x Motivation 0.17" 0.06 0.16" 0.06 0.16" 0.06 
ELL x Motivation -6.13°" 1.44 -6.11°" 1.44 -6.15°" 1.44 
IEP x Motivation -0.04 0.83 -0.03 0.84 -0.05 0.83 
Level 2 (school level) 

Mean motivation 23.02°"* 2.57 16.97" 1.81 
Proportion Black -0.14"" 0.01 
Proportion Hispanic -0.02 0.02 
Proportion ELL -0.17 0.34 
Proportion IEP -1.23™ 0.44 
Proportion NSLP -0.89°** 0.27 
Proportion absent -0.66 0.42 
Mean SES 3.32" 0.25 

Random effects 
Intercept 240.47°* 12.45 210.45°™* 9.93 65.67 4.72 
Slope (motivation) 3.12 1.96 3.15 1.96 3.03 1.95 
Level-1 residual 605.46" 6.00 605.10" 5.98 606.69" 6.02 


*#EE Dy < .001; ** p< .01; * p <.05. 

NOTE: IEP=Individualized Education Program. ELL=English language learners. NSLP=National School Lunch Program. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Assessment of Educational Progress 
(NAEP), 2015 Grade 8 Reading Assessment. 


